Availability of large amounts of raw unlabeled data has sparked the recent surge in semi-supervised learning research.\r\nIn most works, however, it is assumed that labeled and unlabeled data come from the same distribution. This\r\nrestriction is removed in the self-taught learning algorithm where unlabeled data can be different, but nevertheless\r\nhave similar structure. First, a representation is learned from the unlabeled samples by decomposing their data matrix\r\ninto two matrices called bases and activations matrix respectively. This procedure is justified by the assumption that\r\neach sample is a linear combination of the columns in the bases matrix which can be viewed as high level features\r\nrepresenting the knowledge learned from the unlabeled data in an unsupervised way. Next, activations of the labeled\r\ndata are obtained using the bases which are kept fixed. Finally, a classifier is built using these activations instead of the\r\noriginal labeled data. In this work, we investigated the performance of three popular methods for matrix\r\ndecomposition: Principal Component Analysis (PCA), Non-negative Matrix Factorization (NMF) and Sparse Coding (SC)\r\nas unsupervised high level feature extractors for the self-taught learning algorithm. We implemented this algorithm\r\nfor the music genre classification task using two different databases: one as unlabeled data pool and the other as data\r\nfor supervised classifier training. Music pieces come from 10 and 6 genres for each database respectively, while only\r\none genre is common for the both of them. Results from wide variety of experimental settings show that the\r\nself-taught learning method improves the classification rate when the amount of labeled data is small and, more\r\ninterestingly, that consistent improvement can be achieved for a wide range of unlabeled data sizes. The best\r\nperformance among the matrix decomposition approaches was shown by the Sparse Coding method.
Loading....